Method and apparatus for video game simulations using motion capture

ABSTRACT

A method in a video gaming system including a processor, a memory and a sensor system for capturing body motion is described. In one embodiment, the body motion can be associated with a person pretending to hit or launch an object, such as an object used in a sporting activity. In general, body motion can be associated with any activity involving similar body motions that are repeated during the activity. In a video game generated by the video gaming system, a consistency with which the repeated body motions are made over time can be used to determine an outcome for a single instance of the body motion. In a particular embodiment, a probability of a more desirable outcome resulting from the single instance of the body motion can increase as the consistency with which the repeated body motions are made increases.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 13/561,260, filed Jul. 30, 2012, entitled, “METHOD ANDAPPARATUS FOR VIDEO GAME SIMULATIONS USING MOTION CAPTURE,” by Marty etal., which is a continuation-in-part and claims priority to

i) U.S. patent application Ser. No. 12/127,744, entitled “STEREOSCOPICIMAGE CAPTURE WITH PERFORMANCE OUTCOME PREDICTION IN SPORTINGENVIRONMENTS,” filed May 27, 2008, which claims priority to U.S.Provisional Application No. 60/931,950, entitled “STEREOSCOPIC IMAGECAPTURE WITH PERFORMANCE OUTCOME PREDICTION IN SPORTING ENVIRONMENTS,”filed May 24, 2007, and

ii) U.S. Provisional Application No. 61/524,236, entitled, “MOBILENATURAL MOTION CAPTURE DEVICE AND APPLICATION TO VIDEO GAMES,” filedAug. 16, 2011, the contents of each of which are hereby incorporated byreference in their entirety and for all purposes.

FIELD OF THE INVENTION

The present invention is generally related to methods and apparatus forvideo gaming systems. More particularly, the present invention isrelated to methods and apparatus for video gaming systems includingmotion capture.

BACKGROUND OF THE INVENTION

Participating in sporting activities, such as golf, tennis, soccer,basketball and football, and playing video game simulations of thesesporting activities are both popular pastimes. Currently, beyond sharingthe same set of rules, the actual participation in sporting activity isquite different than playing a video gaming simulation of the sportingactivity. For example, the skills needed to actually hit a golf ball arequite different than the skills needed to play a video game simulationin golf.

In recent years, video gaming systems, such as a Wii™ by Nintendo andthe Xbox™ with a Kinect™ sensor by Microsoft have been introduced thatinclude motion sensing capabilities. The video gaming systems and theirsensing capabilities have allowed user motions to be used as an inputmechanism for playing video games. For example, instead of hitting abutton to hit a golf ball in a golf video game, using these new systems,a user can move their body in some manner to hit the golf ball where howthey move their body can affect the input used for the video game andhence the golf shot shown in the video game.

One reason sports video game simulations are popular is because of theirrelationship to the actual activity. Although the new video game systemsnow allow for limited user motions while the video game is played, therelationship between the motions made while playing video games andwhile playing the actual sports activity is at best superficial. In viewof the above, new methods and apparatus for are desired using motionsensing capabilities for playing video games.

SUMMARY OF THE INVENTION

A method in a video gaming system including a processor, a memory and asensor system for capturing body motion is described. In one embodiment,the body motion can be associated with a person pretending to hit orlaunch an object, such as an object used in a sporting activity. Themethod can be generally said to include: 1) receiving in the processor amotion consistency parameter where the motion consistency parameter isgenerated from data captured during a plurality of different instancesof the person attempting to repeat the body motion and where adetermining of the motion consistency parameter includes determining aposition of at least one point on a body of the person during each ofthe plurality of different instances of the body motion and determininga standard deviation using the determined positions; 2) capturing viathe sensor system data for a single instance of the person attempting torepeat the body motion while pretending to hit or launch the object; 3)based upon the data from the single instance, in the processor,predicting a magnitude of a force and a direction of the force thatwould be imparted to the object if it was actually hit or launchedwherein physical properties of the object including its size and massare simulated; 4) determining whether the magnitude of the estimatedforce and the direction of the force are within acceptable ranges for avalid body motion associated with the person pretending to hit or launchthe object; 5) when the magnitude of the estimated force and thedirection of the estimated force are determined to be within theacceptable ranges, determining in the processor whether a desiredoutcome for the simulated object is possible; 6) when the desiredoutcome is determined to be possible, determining in the processorwhether the desired outcome or a non-desired outcome has occurredwherein a probability of the desired outcome occurring increases as thestandard deviation associated with the motion consistency parameterdecreases; and 7) rendering in the processor to a display a simulatedtrajectory of the simulated object for the single instance, wherein therendered simulated trajectory shows the determined desired outcome orthe determined non-desired outcome occurring for the simulated object.

In general, body motion can be associated with any activity involvingsimilar body motions that are repeated during the activity. In a videogame generated by the video consistency, a consistency with which therepeated body motions are made over time can be used to determine anoutcome for a single instance of the body motion. In a particularembodiment, a probability of a more desirable outcome resulting from thesingle instance of the body motion can increase as the consistency withwhich the repeated body motions are made increases.

As an example, in a boxing game, a person may repeatedly strike out withtheir arm to simulate a punch or raise their arm to simulate a block. Amotion consistency parameter can be generated from data captured duringa plurality of different instances of the person attempting to repeatthe punching or the blocking body motions. Then, the motion consistencyparameter can be used to determine an outcome of a single instance of abody motion including a punch or block, such as whether the punch landsor misses and whether the block is successful or not. In particular, themotion consistency parameter can affect a probability of the punchlanding or the block being successful, such as increasing or decreasingthe probability depending on its value.

In particular embodiments, the object the person is pretending to hit orlaunch is a ball. The ball can be one of a basketball, tennis ball,volleyball, golf ball, baseball, soccer ball, bowling ball or afootball. As an example, the person can be pretending to shoot abasketball where the desired outcome is a made shot and the non-desiredoutcome is a missed shot in a video game generated by the video gamingsystem.

In other embodiments, the desired outcome can be the simulated objectstopping within a defined area and the non-desired outcome can be thesimulated object stopping outside of a defined area in a video gamegenerated by the video gaming system. In one example, the video game canbe a golf video game and the desired outcome is one of a simulated golfball coming to rest at an optimal distance in a fairway or on a greenand the non-desired outcome is one of the simulated golf coming to restoff the fairway, off the green or at a non-optimal distance on thefairway or the green. The desired outcome can also be the simulatedobject passing through a defined area and the non-desired outcome is thesimulated object passing outside of the defined area in a video gamegenerated by the video gaming system. For example, the video game can bea soccer video game where the defined area is a plane of the goal. Thedesired outcome can be the soccer ball passing through the plane of thegoal and the non-desired outcome is the soccer ball passing outside theplane of the goal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a bar graph of a percentage of basketball shots made versusconsistency score in accordance with an embodiment of the presentinvention.

FIG. 1B is a bar graph of numbers of players with consistency scores invarious ranges as determined from basketball data in accordance with anembodiment of the present invention.

FIG. 2 is a chart illustrating the effects of release velocity versusrelease angle for basketball in accordance with an embodiment of thepresent invention.

FIG. 3 is flow chart of a method for determining an outcome for a videogame in accordance with an embodiment of the present invention.

FIGS. 4A and 4B are diagrams of a basketball shooter near a beginningand end of a basketball shot in accordance with an embodiment of thepresent invention.

FIG. 5 is a perspective drawing of a video gaming system including amotion capture sensor in accordance with an embodiment of the presentinvention.

FIG. 6 is a block diagram of a video gaming system configured to predictoutcomes using motion consistency parameters in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION

In the following paper, numerous specific details are set forth toprovide a thorough understanding of the concepts underlying thedescribed embodiments. It will be apparent, however, to one skilled inthe art that the described embodiments may be practiced without some orall of these specific details. In other instances, well known processsteps have not been described in detail in order to avoid unnecessarilyobscuring the underlying concepts.

Typically in sports, certain motions are repeated again and again aspart of playing the game where the motions are relatively similar frominstance to instance. For example, in basketball, a player shoots abasketball, such as free throw, or takes other shots where a somewhatsimilar shooting motion is repeated for each shot. In baseball, apitcher throws a baseball to the catcher and a batter swings a bat,where the throwing or swinging motions are similar for each throw orswing. In golf, a player swings different clubs or putts to hit a golfball, where the swinging motion for each swing or putt is somewhatsimilar. In tennis, a player serves the tennis ball or chooses from anumber of different strokes to hit the tennis ball. There aresimilarities in the serving motion from serve to serve or in the strokemotions for each type of stroke.

In football, a quarterback throws the football or a kicker kicks thefootball, where the throwing motions or kicking motions are similar formany of the throws or kicks. In soccer, a player kicks the ball, wheremany of the kicks and the kicking motions are similar. For example, foreach instance of a corner kick, the kicking motion can be similar fromkick to kick. In volley ball, a player serves, sets or spikes thevolleyball, where there can be similarities between different instancesof these motions.

In each of the sports above, under a variety of conditions a player istrying to control a placement of an object, such as a ball. Theplacement of the object can involve controlling the trajectory of theobject so that it travels through some bounded area or arrives withinsome volume of space at a desired time. For example, for a field goalkicker, the objective is for the ball to pass through area between thegoal posts along its trajectory. As another example, for a quarterbackor a soccer player, it is desirable to place the ball in a particularvolume of space at the same time another player is entering the space.In yet another example, a hockey player tries to place the puck througha plane in front of net not blocked by a goalie.

The control of the placement of the object can also involve attemptingto control the final resting point of the object after it has been hitor launched. For example, in golf, it may be desirable to place the golfball in a fairway as opposed to the rough. As another example, it may bedesirable to hit the ball so that it lands on the green as opposed to ahazard around the green, such as the rough, a water hazard or a bunker.

Many of these situations are repeated and the objective of the playereach time the situation occurs is to perform an action with their bodiesthat achieves the desired outcome. Generally, the player strives toreproduce the same motion each time. However, because of the limits ofthe human body, as any person that has watched professional basketballplayers shooting free throws can attest, the ability to reproduce thesame motion each time within the range of accuracy needed for aparticular sport is easier said than done. Nevertheless, the process ofovercoming the difficulties associated with repeatedly moving one's bodyin a particular way to allow success in a sport is what makes the sportinteresting to most individuals.

For all these sports, many theories have been developed in regards towhat makes a particular player skilled, what optimal techniques to useand how any player can develop these skills. Typically, the theories aredeveloped by looking at a few individuals that are deemed successful intheir sport, determining what motions they make and then specifyingtraining techniques that allow another individual to mimic the motions.The success of this approach is limited because when it comes tocontrolling errors associated with body motions every individual isdifferent. Thus, the techniques that work for one skilled individualdon't apply to every individual and it is difficult to determine foreach individual what techniques to utilize.

What has not been done in the past is a rigorous analysis in sports,such as but not limited to basketball, of how a consistency of anindividual's ability to repeat body motions associated with the sport isrelated to their skill level where the analysis is based upon datameasured from a large number of individuals with a wide range of skilllevels and body types. Thus, with respect to FIGS. 1A, 1B and 2, ananalysis of skill level as related to a reproducibility of body motionsfor basketball, where a large number of players with varying skilllevels are considered, is discussed. One observation from the analysisis that players that are most consistent in making the same shootingmotion from shot to shot also make the highest percentage of theirshots. As will be discussed in more detail below, this observation seemsto be independent of technique.

As is described with respect to FIGS. 3, 4A, 4B, 5 and 6, a method forvideo game simulations that utilizes this observation about consistencyof body motion is described. In particular embodiments, the method isutilized in video gaming systems including body motion detectioncapabilities, such a Xbox360™ system from Microsoft™ with a Kinect™sensor. The method can be configured to utilize body motion datacaptured from a single instance of a body motion associated withpretending to hit or launch an object in a sports video game and theconsistency at which the body motion has been repeated from a number ofprevious different instances to determine the outcome resulting from thesingle instance of the body motion in the sports video game.

As an example, in a basketball video game, an individual can make thebody motion of pretending to shoot a free throw. A sensor systemassociated with the video gaming system can capture aspects of theshooting motion. Based upon data derived from the captured free throwmotion and the consistency with which the individual has been able toreproduce the shooting motion based upon a number of differentpreviously captured measurements of the person making the shootingmotion, a prediction can be made in regards to the outcome of the mostrecently captured shooting motion, such as whether the free throw ismade or missed. Once the outcome to the free throw is determined, avideo animation of the shooting motion can be rendered and output to adisplay. The video animation can include an individual shooting and thebasketball travelling a trajectory that shows the determined outcome. Asdescribed above and in accordance with actual shooting data, the methodcan proscribe that individuals more able to consistently reproduce asimilar shooting motion will on average make more shots in the videogame as compared to individuals that are less consistent at reproducingtheir shooting motions.

Motion Consistency as a Performance Predictor

In this section, motion consistency as a performance predictor isdescribed. In particular, shooting motions related to basketball and theobservation that individuals whose shooting motions vary less from shotto shot tend to make a higher percentage of their shots is discussed.For basketball, the motion consistency can be the consistency of theshooting motion from shot to shot and the performance predictor may bethe percentage of shots made. Other body motions and performancepredictors associated with sporting activities other than basketball canbe analyzed and the example of the shooting motion in basketball isprovided for the purposes of illustration only.

The consistency observations for basketball are based upon an analysisof a large amount of data gathered from individuals actually shootingbaskets and a mechanical device constructed to shoot baskets and mimicthe variations in shooting motions which are exhibited by individuals.Details of the mechanical shooting device and other factors related topredicting performance are described in U.S. application Ser. No.12/127,744, previously incorporated herein. Some details of theexperimental data are described as follows.

FIG. 1A is a bar graph of a percentage of basketball shots made versusconsistency score and FIG. 1B is a bar graph of numbers of players withconsistency scores in various ranges as determined from basketball data.To generate the figures, data was gathered from around a thousanddifferent individuals shooting basketball free throws. On the order ofat least 10 shots were measured for each player. Thus, the data setincludes measurements from more than 10,000 shots.

Video images of a shot basketball travelling along its trajectory werecaptured for the different individuals and their different shots. Thetrajectories of the shot basketballs for each individual werecharacterized using the video data. One example of a system that can beused to characterize the trajectories is the Noah™ select system byPillar Vision, Inc. (Athens, Ala.). Some details related tocharacterization of trajectories of objects in various sportingenvironments are described in U.S. Pat. Nos. 7,094,164, 7,850,552,7,854,669 and U.S. patent application Ser. Nos. 11/972,553, 12/015,445,12/127,744, 12/966,301 each of which is incorporated by reference andfor all purposes.

In the examples of FIGS. 1A and 1B, an entry angle of the basketball asit approaches the basketball hoop and a location that the center of theball crosses the plane in of the hoop are characterized for each shot byeach individual. Variations in these parameters reflect variations inshooting motions of the individual from shot to shot. If the individualshoots the ball with the exact same shooting motion each time, then froma given shot location, such as the free throw line, the entry angle andthe location that the center of the ball crosses the plane of the hoopwould be the same each time.

Based upon the shooting data analyzed for each individual, a consistencyscore was generated. The consistency score is one example of aconsistency parameter that can be used to characterize a consistency ofbody motion that is repeated during an activity, such as a sportingactivity. The consistency score for a number of shots can configured asa function of i) the entry angle, standard deviation and ii) a distance,standard deviation. The distance used in the standard deviation can bethe location where the center of the ball crosses the plane of the hoop.The minimum value of the standard deviation of the entry angle or thestandard deviation of the distance is zero.

In statistics and probability theory, standard deviation (represented bythe symbol σ) shows how much variation or “dispersion” exists from theaverage (mean, or expected value). A low standard deviation indicatesthat the data points tend to be very close to the mean, whereas highstandard deviation indicates that the data points are spread out over alarge range of values. Other examples of measures that can be used tocharacterize an amount of variation in a data set and which can beapplied to generate a consistency parameter are a range and a variance.

The consistency score can be configured to have a maximum value of 10.The maximum value of 10 can be generated for a shot data set where theentry angle for each shot and the location that the center of the ballcrosses the plane of the hoop are the same for each shot. For such adata set, the standard deviation of the entry angle and the standarddeviation of where the ball crosses the plane of the hoop are both zero.Hence, the consistency score is ten.

When the entry angle and the location that the center of the ballcrosses the hoop plane are the same for two trajectories, it doesn'tnecessarily mean that the two trajectories are exactly the same. It onlymeans that these two values derived from the trajectory analysis of eachtrajectory are the same. Additional or other trajectory parameters canbe derived and can be used to characterize a trajectory of an objectwith more refinement if desired.

For a group of trajectories, these two parameters provide a reasonablyrepresentation of how consistent each of the trajectories are in thegroup and hence how consistent the body motions were that generated thetrajectories. However, as will be described in more detail below, otherparameters can be used to characterize the consistency of body motionsover a number of repeated attempts. For example, the motion of the body,such as the motion of a hand or an elbow during a basketball shot, canbe measured directly and motion consistency parameters can be derivedfrom these measurements.

The direct measurement of a body motion can be used in lieu of or incombination with a measurement of a trajectory of an object launched orhit as a result of the body motion. For example, a first motionconsistency parameter can be defined which utilizes a standard deviationassociated with measured trajectories of objects launched or hit as aresult of the body motion and which utilizes one or more standarddeviations associated with measurements at one or more points on thebody while the individual hits or launches objects. A second motionconsistency parameter can be defined which only utilizes one or morestandard deviations associated with measurements at one or more pointson the body while the individual launches or hits objects.

The second motion consistency parameter can be used for characterizing aconsistency of motions when objects, such as a ball, are not launched orhit. For example, the individual might pretend to hit or launch a ball anumber of times and the consistency of these motions can becharacterized. As another example, the individual might make motionsassociated throwing a punch or making a kick and the consistency ofthese motions can be characterized.

Returning to the example of basketball, in FIG. 1A an average shootingpercentage for a free throw for individuals with consistency scores invarious ranges are shown. In FIG. 1B, an amount of individuals in eachconsistency score range are shown. As shown in the FIG. 1A, the averagepercent of shots made increases as the consistency score increases. Asdescribed above, the consistency score provides a reasonable measure ofan individual's consistency at making their shooting motion i.e., haveless variability in their shooting motion from shot to shot. Thus, itcan be concluded that individuals that are more consistent in makingtheir shooting motion make a higher percentage of their shots and hencecan be considered more skilled shooters.

Although not shown, factors related to technique were considered. Forexample, it has been found that some arcs are more optimal for making ashot than other shots. Thus, individuals that shoot a ball with a moreoptimal arc increase their chances of making a shot. However, from thedata represented in FIGS. 1A and 1B, individuals were grouped that shotthe ball on average with a similar arc. When the individuals weregrouped in this manner, it was still found that individuals with ahigher consistency score made a higher percentage of their shots ascompared to individuals with a lower consistency score. Other effectsassociated with technique were also considered and again, therelationship that individuals with a higher consistency score made ahigher percentage of their shots still held.

From these results, it was concluded that independent of technique,individuals that are more consistent at making their shooting motionsuch that their variations in their body motions from shot to shot arelower as compared to other individuals can be said to be more skilledthan individuals that are less consistent in making their shootingmotions. The greater shooting percentage of individuals with a moreconsistent shooting motion backs this skill level conclusion. Ingeneral, for many different activities involving a repeated body motion,it is believed that independent of technique, individuals that are moreconsistent at making a repeated body motion such that their variationsover each of the instances of the body motions are lower as compared toother individuals, can be said to be more skilled at the activity thanindividuals that are less consistent at making their repeated bodymotions.

As will be discussed in more detail as follows, these conclusions areapplied to develop a family of video games that use a characterizationof the consistency of repeated body motions to affect a prediction of anoutcome resulting from a single instance of the body motion. These videogames may be played on video gaming systems including the capability tocapture body motions. An Xbox-type video gaming system from Microsoft™with a Kinect™ sensor is one example of system on which the video gamescan be played.

A few examples of how these motion capture capabilities can be utilizedin a video game are described as follows. In a basketball video game, ameasured consistency of a shooting motion of a person pretending toshoot a ball can be used to affect a prediction of whether one instanceof the person pretending to shoot the ball will be made or missed. Thedetermined outcome can be rendered as part of the video game. In a golfvideo game, a measured consistency of a person pretending to swing aclub many different times can be used to affect a prediction of whetherone instance of the person pretending to swing the club results in adesirable shot or a non-desirable shot, such as on the fairway or in therough. In a fighting video game, a measured consistency at which aperson makes a punch or a kick over many different instances can be usedto affect a prediction of whether one instance of a punch or a kick hitsor misses an opponent in the video game. It is believed that these typesof games will be more enjoyable than existing video games becauseimprovements in consistency of body motion in games involving motioncapture will result in better outcomes in video games in the same waythat improvements in consistency of body motion result in betteroutcomes in actual sporting activities as illustrated above with respectto the example of basketball.

Outcome Prediction

As described above, a measure of consistency of a motion can be used toaffect a prediction of an outcome resulting from one instance of themotion in a context of a video game. Thus, prior to discussing detail ofa method for video games in FIG. 3, a few details of outcome predictionare discussed with respect to FIG. 2. In particular, a few detailsrelated to outcome prediction for basketball are described as follows.

FIG. 2 is a plot showing combinations of release velocity and releaseangle that result in a made or a missed free throw based upon computersimulations of free throws. The free throws are assumed to travel alonga trajectory that is perpendicular to the backboard and bisects thehoop, i.e., there are no side to side errors. The release anglerepresents an angle at which the ball is released from a shooter's hand.The release velocity is related to an amount of force with which thebasketball is shot. Six bands, 108, 110, 112, 114, 116 and 118 are shownin FIG. 3. Shots with a combination of release velocity and releaseangle within the bands result in a made shot. Shots with a combinationof release velocity and release angle outside of the bands are missedshots.

In more detail, band 108, which is narrow, is associated with shots thathit the front of the rim and then go through hoop. Band 110 which is thewidest band when the release angle is about 50 degrees is associatedwith swish shots, i.e., shots that pass through the hoop without hittingthe rim. Band 112 and 114 represent shots that hit the back of the rimand go in. Band 116, which is the second largest band, is associatedwith bank shots that swish after hitting the backboard. Finally, band118 represents shots that hit the backboard, the front of the rim andthen goes through the hoop.

A few examples of shot trajectories and their corresponding releaseangle and release velocity are shown. Trajectory 102 represents a madeswish shot. Trajectory 106 represents a made bank shot that hits thebackboard and swishes through the hoop. Trajectory 104 represents amissed shot where the basketball travels over the backboard.

Most games of skill like basketball are designed to be sensitive tosmall changes in body motion. For example, a small change in the releasedirection and force can result in a made or missed shot. When utilizingmotion capture capabilities available with current video gaming systems,a force and a direction that might be imparted on an object as a resulta body motion can be estimated with some level of precision. Typically,the precision level of the estimation is isn't great enough to preciselypredict an outcome. For example, a person can pretend to shoot abasketball and the force that would be imparted to an actual basketballand the trajectory of the basketball after the force is imparted can beestimated. However, the error levels in the estimation may be too largeto predict with certainty whether the shot is made or missed.

As an example, in FIG. 2, an error ellipse 120 that accounts for errorsin release angle and the release velocity that might be estimated frommotion capture data from a motion capture sensor associated with acommercial video gaming system is shown. The error ellipse is centeredabout trajectory 102 which is swish shot. Within the error ellipse arelarge number of outcomes are possible that can be represented bydifferent trajectories. The possible outcomes within the error ellipseinclude 1) missed shots, 2) shots that hit the front of the rim and goin, 3) shots that swish and 4) shots that hit the back of the rim andshots that hit the back board and go in. Graphically, the probability ofa shot being made is the area of the error ellipse associated with madeshots divided by the total area of the error ellipse.

In a particular embodiment, to determine whether a force and directionestimated from motion capture data results in a particular outcome, aprobabilistic approach can be utilized. In the probabilistic approachfor a given force and direction estimated from the motion capture data,probabilities can be determined for each of two or more possibleoutcomes that can result from the body motion that is captured. Then, arandom number can be generated to select one of the possible outcomesbased upon the determined probabilities. For example, for a personpretending to shoot a basketball, a first probability can be determinedfor whether the shot is made and a second probability can be determinedfor the shot being missed. As another example, for a golf game, a firstprobability can be determined for whether a shot lands in the fairway, asecond probability can be determined for whether the shot lands in therough and a third probability can be determined for whether the shotlands in a hazard, such as a bunker or water.

As described above, the probability of a desirable outcome can beincreased for a single instance of a body motion based upon a measure ofconsistency of the body motions that the individual has previously made.Using the basketball example, a probability of a shot being made can beaffected by a measure of a consistency of the individual's shootingmotions for a number of previous shots. As an example, the probabilityof a made shot can be increased as a value of the consistency score (seeFIGS. 1A and 1B) is increased. As previously described, it was observedthat the consistency score increases as the skill level of the shooterincreases. Using the golf example, the probability of the ball coming torest in the fairway can be increased when an individual is moreconsistently able to repeat the swinging motion.

A more detailed example of this probability formulation is described asfollows with respect to basketball. Similar formulations can be definedfor predicting outcomes involving body motions associated with othersporting and non sporting activities. Thus, the basketball example isprovided for the purposes of illustration only and is not meant to belimiting.

First, based upon a person pretending to shoot a basketball a forceincluding its magnitude and direction can be estimated that would beimparted to the basketball if the person where actually shooting thebasketball. The estimated force can be derived from the capturedshooting motion data. Based upon the estimated force, a closest distanceof the ball to backboard or the hoop along its trajectory can beestimated. In one embodiment, a shooting motion which predicts that thebasketball doesn't come within a first threshold distance from thebackboard or rim, such as twenty feet from the rim or backboard, may notbe considered a shot and thus may not presented to the player with aparabolic ball flight when the results of the shot are rendered in thevideo game. Shots predicted from body motions of this type have zeroprobability of scoring.

Some of these types of shots can be identified by ranges specified formagnitude and direction of the estimated force. For example, anglesranges can be specified that define invalid shots. The range mightinclude shots that appear to be shot straight up, directly into theground or even backwards. When one of these shots is identified basedupon a comparison to the ranges, the video game may render anon-shooting outcome rather than a shooting outcome. For example, thevideo gaming system might display a turnover, such as the ball beingthrown out of bounds, stolen or blocked or some other non-desirableoutcome.

Body motions that predict a simulated basketball is to come within arange, such as between five and twenty feet, of the rim or backboard maybe considered a shot. The shot may be presented to the player with aparabolic ball flight which misses the backboard and rim. These shotsalso have a zero probability of scoring.

Body motions that predict a simulated basketball is to come within arange of the backboard or rim, such as within five feet may beconsidered a shot and can be presented to the player with a parabolicball flight. The shot can be given some probability of being a madeshot. In this example, the shot is given a 60% probability of scoring.

A determination of whether the shot is made or not can be generatedusing a random number. For instance, a random number between 0 and 1 canbe selected. If the generated number is less than or equal to 0.6, theshot is considered made. If the generated random number is greater thanor equal to 0.6, the shot is considered missed.

For a made shot, the ball flight that is presented can be mostlyconsistent with the direction of the force and the magnitude of theforce as originally estimated from the player's natural motion. However,the ball flight may be modified slightly to align with a variety of madeshots possibilities, such a swish, a rim in or a bank in. For a missedshot, the ball flight presented may be largely consistent with theestimation of the direction of the force and the magnitude of the forceas originally predicted from the player's natural motion. However, theball flight may be modified slightly to align with a variety of missedshots possibilities, such as off the rim, in and out or an air ball.

The method above provides an example of a baseline calculation that canbe utilized when a measure of the consistency of the individualsshooting motion is not available. For example, the video gaming systemmay be configured to generate a new motion consistency parameter thefirst time a person starts playing and then may also be configured toallow a user to reset the motion consistency parameter for some time totime. For some number of shots, the system may be configured not to usea motion consistency parameter until enough data is gathered to generatethe new motion consistency parameter. In one embodiment, the system mayutilize a new motion consistency parameter after data from as little astwo body motions associated with shots have been measured.

Next an example of outcome shot prediction is provided that accounts fora measure of motion consistency. In one embodiment, as will be describedin more detail with respect to FIGS. 4A and 4B, a motion consistencyparameter can be defined as an angle involving three body points. Forexample, an initial body angle (IBA) can be defined as an angle betweena line from the individuals guide hand elbow to their shooting handelbow and a line from the individuals guide hand elbow to the knee ontheir shooting hand side. The shooting hand elbow can be the person'sleft or right elbow depending on whether they are right handed or lefthanded.

Next, the individual can be asked to take i) a number of actualbasketball shots, ii) a number of pretend basketball shots or iii) acombination of actual and pretend shots. The pretend shots can bewithout the ball. In one embodiment, a user can be asked to take and/orpretend to take about 25 shots. When the actual shots are taken, theshots can be from a particular distance, such as from the free throwline or from the three point line. Data from the body motions can becaptured using a sensor associated with a video game system and/orsensors separate from the video game system. For example, data may becaptured and analyzed using a NOAH™ system (Pillar Vision, Inc.) thatmay exported to a video game system, such as an Xbox-type system with aKinect™ sensor from Microsoft™.

As described above, the captured motion data can be analyzed todetermine an initial body angle (IBA) for each of the actual shots orpretend shots. Then, a standard deviation for the IBA can be determinedUsing the IBA, a shooting percentage for the individual can be generatedas a function of the IBA standard deviation. For example, the shootingpercentage can be generated as,

shooting percentage=(−15*IBA standard deviation)+97,

In this example, the values in the formula were generated by measuringthe IBA standard deviation and the associated shooting percentages forvarious individuals shooting actual shots and then correlating theformula to the empirical data. A line was used to fit the empiricaldata. However, in other embodiments, a more complex curve fit can beused to fit the data.

In particular embodiments, IBA can determined for shooting motions withand without the ball. The standard deviations of the IBA for shootingmotions with and without the ball may be different. In one embodiment, acorrection factor can be developed for the IBA standard deviation fromshots without the ball before it is used in the shooting % formula abovethat is correlated to empirical data. For instance, when an individual'sIBA standard deviation for shooting motions measured without the ball istypically 20% higher than the standard deviation for their measuredshooting motions with the ball. The IBA standard deviation for shootingmotions without the ball can be reduced by 80% before being applied to aformula that was correlated to match measured shooting percentages fromactual shots.

As will be described in more detail with respect to FIGS. 4A and 4B,many different types measurements can be used to generate motionconsistency parameters and these parameters may also be adjusted toaccount for standard deviation derived from motions with and without anobject. For example, for golf, motion consistency parameters can becorrelated to account for standard deviations derived from motions withand without a golf club. For tennis, motion consistency parameters canbe correlated to account for standard deviations derived from motionswith and without a tennis racket.

Returning to the basketball example, the shooting % determined via theformula [shooting %=(−15*IBA standard deviation)+97] can be used withthe outcome prediction methodology described above where a fixed valueof 60% was specified as the probability of the shot being made if theshooting motion indicated the shot would approach within 5 feet of thebasketball hoop or backboard. The shooting percentage calculated fromthis formula can be used instead of the 60%.

In this example, as the IBA standard deviation decreases, i.e., theperson is more consistent at repeating the shooting motion, the person'schances of the predicted outcome of the shot being a made shotincreases. The system can be configured to recalculate the standarddeviation used in a probability formulation, such as the IBA standarddeviation used to determine a probability of making a shot, so that itchanges over time. For example, the standard deviation of the shootingmotion can be calculated based upon some number of the most recentshooting motions considered shots (As described above, some shootingmotions may not be considered as valid shots by the system.). Thus, likein real-life, if a person gets tired and their motions become lessconsistent, the likelihood of the person making a shot in the video gamewill also decrease.

In various embodiments, standard deviations used in an outcomeprediction formulation can change over time as different instances of abody's motion are captured and analyzed by the video gaming system.Further, the video gaming system may allow a person to save informationassociated with the determination of measures of motion consistency,such as standard deviations, so that data used in a first video gamingsession can carry over into a second video gaming session. In addition,the video gaming system may be configured to allow a person to import orexport data used in the determination of measures of motion consistency.For instance, import/export capabilities may allow two individuals tocompete against one another in a video game where data associated withthe determination of motion consistency is generated and exported from afirst video gaming system and imported to second video gaming system.After the data is imported to the second system, motion consistencyparameters can be generated for both individuals and a video gameinvolving a competition between the two players can be initiated.

In one embodiment, the shooting percentage formulation that is afunction of a motion consistency parameter can be used to affect theoutcome resulting from the body motion that is depicted in the videogame. As described above, due to the fidelity of the motion capturesystem, many different outcomes can be consistent with the force and thedirection of the force that is estimated from an individual's bodymotions captured by the system. In particular embodiments, depending onthe consistency of a person's body motions over many differentinstances, a selection of an outcome for a particular instance that isto be depicted can depend on a motion consistency parameter.

For example in a basketball video game, if a person achieves a shootingpercentage above a first threshold, such as 90% or greater, based uponthe consistency of their shooting motions, all of the outcomes predictedto be made shots can be depicted in the video game as swish shots orhitting the back rim and going down. For a second range of shootingpercentages, such as between 90% and 70%, made shots can be depicted asa swish, a back rim and down or front rim and up. For a third range ofshooting percentages, such as between 70% and 50%, made shots can bedepicted as swish, back rim and down, front rim and up, and include anoccasional roll around the rim. For a fourth range, such as a shootingpercentage less than 50%, made shots can be depicted as swish, back rimand down, front rim and up, a roll around the rim, or an occasional bankshot.

In one embodiment, different probabilities of each of the outcomesoccurring can be assigned and then selected using a random number. Forexample, for an individual with a shooting percentage greater than 90%,probabilities can be determined for each the outcomes of a swish shot ora shot hitting the back rim and going down occurring. Then, one of theseoutcomes can be selected based upon the determined probabilities. Asanother example, for an individual with a shooting percentage less than50%, probabilities can be determined for each of a swish shot, a backrim and down shot, a front rim and up shot, a roll around the rim shotor a bank shot. Based upon the probabilities, one of the outcomes can beselected for the depiction of a made shot in the video game.

The system can also be configured to depict missed shots differentlydepending on the shooting percentage determined using a motionconsistency parameter. For example in a basketball video game, forshooting percentages greater than 90% as determined by the formuladescribed above, all missed shots can be depicted as traveling straightand hitting the front rim or the back rim during a miss. For shootingpercentage between 90% and 70%, missed shots can be depicted as hittingthe front rim, back rim or side rim. For shooting percentage between 70%and 50%, missed shots can be depicted as hitting a front rim, a backrim, a side rim and occasionally rolling around the rim and out. For ashooting percentage less than 50%, missed shots can be depicted ashitting a front rim, a back rim, a side rim, rolling around the rim andout, a bank off the backboard followed by a hit or miss of the rim, oran air ball.

In basketball, shots can be taken from different distances. Typically,an individual's shooting percentage decreases as a distance from thehoop increases. In various embodiments, probabilities for outcomesresulting from body motions can be adjusted to account for factors suchas distance. For instance, an outcome prediction formula of shootingpercentage=min [(−15*IBA standard deviation−(D−11)), 0]+97 can be usedwhere “D” is a distance measured in feet from the basket to account forvarious shooting distances. The IBA is the initial body angle describedabove.

The outcome prediction formula in the previous paragraph may have beenfirst correlated to actual shooting data and body motion analysisassociated with shots of eleven feet. Thus, at distances of eleven feet,the shooting percentage is not affected by distance. As the distancefrom the basket approaches zero, the shooting percentage is increased asa result of being closer to the basket. The maximum possible shotpercentage is 97%. As the distance from the basket increases, theshooting percentage including the maximum shooting percentage is reducedas result of the distance. If empirical data is available for shots fromdifferent distances where body motions have been characterized at eachdistance, then curve fits can be developed that account for the effectson the shooting percentage of both distance and the consistency of bodymotion. The curve fits can be used to determine a probability of a shotoutcome as a function of motion consistency and distance from the basketin a video game.

In other types of games more than one motion can be utilized during aplay of the game. For example, in a tennis video game at a first time,an individual can pretend to hit a serve, at a second time, theindividual can pretend to hit a ground stroke and a third time anindividual can pretend to hit an overhead shot. For each of the motions,probability formulations can be developed for predicting an outcomeresulting from the motion, such as the shot being in or out. Theprobability formulations can vary according to a consistency with whichthe motion has been previously made as determined by data captured froma motion capture system.

Video Game System and Method Incorporating Motion Consistency Parameters

FIG. 3 is a flow chart of a method 300 for determining an outcome for avideo game played on a video gaming system. In 302, a number ofdifferent instances of a body motion associated with a particularactivity can be characterized. The characterization can involvecalculating one or more different motion consistency parameters. Thecharacterization may involve the video gaming system. For instance, thebody motions may have been characterized in a previous video gamingsession on the video gaming system.

In 304, the video gaming system can be set-up and a game can beinitialized. The set-up of the video gaming system may involveconfiguring a motion capture device. In 306, a single instance of bodymotion associated with a repeatable activity can be captured using themotion capture system and characterized. Example of a body motion thatmight be captured include but are not limited to a person pretending tothrow, kick, hit or launch a object, such as a ball.

In 308, an outcome resulting from a characterization of the singleinstance of the body motion can be predicted. In 310, the system candetermine if consistency data is available. The consistency may be datarelated to previously made body motions that can be used to determineone or more motion consistency parameters and/or may be one or morepreviously determined motion consistency parameters.

When no consistency data is available, in 314, the outcome determined in308 can rendered. As described above, an outcome can be depicted in anumber of different ways. Thus, the system can be configured to select adepiction of an outcome to render. The depicted outcome can be output toa display as a video animation.

In 310, when consistency data is available, a motion consistencyparameter can be determined The motion consistency parameter cancharacterize the consistency with which a number of different instancesof the body motion were made. The number of different instances of thebody motion and the single instance of the body motion captured in 306will typically be similar to one another.

In 312, an outcome can be determined based a single instance of the bodymotion and one or more motion consistence parameters. In one embodiment,probabilities of one or more outcomes can be determined where theprobabilities are affected by a value of the one or more motionconsistency parameters. Then, based upon the determined probabilities, aparticular outcome can be determined The determined outcome can berendered again in 314. Again, the system can be configured to select adepiction of an outcome to render. When consistency data is available,the selection of the outcome to depict can depend on the consistencydata.

In 316, a decision can be made whether to update the consistency data.For example, if the single instance is determined not to be valid forsome reason, then it may not be added to the consistency data. However,if the single instance is valid, it can be added to the consistency datain 318 and used to determine one or more new consistency parameters whenthe next instance of the body motion is captured and characterized. Whenthe single instance is determined not to be usable for some reason, thenthe consistency data may not be updated.

As described above, a characterization of the consistency of a bodymotion can be based upon identifying one or more different locations onthe body using a motion capture system. After the locations areidentified, one or more motion consistency parameters can be generatedbased upon data associated with the one or more points. The process ofidentifying points on the body and then generating a motion consistencyparameter is described in more detail with respect to FIGS. 4A and 4B.

In FIG. 4A diagrams of a body of a basketball shooter near a beginning300 a and end 300 b of a shooting motion associated with a basketballshot is shown. A number of locations are identified on the individual'sbody. These locations include a top of the head 302, the nose 303, anintersection between the neck and the body 308, where the arms attach tothe body 304 and 310, elbows 306 and 312, wrists 314 and 315 and knees316 and 318.

In 300 a, the individual is shown holding a basketball 320. In 300 b,the individual has performed a shooting motion and released thebasketball 320. As described above, an individual may make a motionwhere they are pretending to perform an activity, such as shooting abasketball. When pretending to shoot a basketball, the individual mightplace their hands in a position as if they were holding a basketball andthen make their shooting motion without actually holding a basketball.

With a video game system, it may be desirable for the individual topretend to make a motion, such as a motion involving hitting orlaunching an object, because they may be in an environment where hittingor launching an actual object would cause damage. For example, if anindividual is playing a video game in their living room, shooting a realbasketball would likely be destructive. However, it may still bepossible to utilize objects with the system that embody some aspect ofan actual object that might be used during an activity but have lesspotential for destruction.

As an example, the basketball 320 might be an object that it is the samesize of an actual basketball but much lighter and/or softer. Thus, theindividual may be able to make the motion of shooting a basketballtowards a basket while holding the object and even let go of the objectwithout causing damage. For basketball, the use of the lighter butbasketball-sized object can help the person to maintain the same bodyorientation as if they were shooting an actual basketball, such as thecorrect spacing between the hands that occurs when an individual holds abasketball. Thus, it may be easier to generate a pretend shooting motionthat is closer to the shooting motion that is generated when a person isactually shooting a basketball.

In another example, for a golf game, an individual might use a handlethat is about the same size as a golf club grip. The handle can belight, such that if the individual accidently releases it, it will notcause damage. The use of the handle allows the individual to place theirhands in correct grip position for a golf swing which may make easier toimitate the body motion of swinging of an actual golf club.

Different algorithms exist for identifying body parts and identifyingthe linkages between the body parts from image data and then mapping thebody parts to a skeletal system. The skeletal system may be a simplestick figure that joins various identified parts at the identifiedlinkages. The system can then track the motions of the different partsas they move according to the identified linkages and adjust theskeletal system accordingly.

The Kinect sensor uses an infrared light to generate a depth map thatcan be associated with the image data. Using the depth map, one or moreindividuals can be distinctly identified in the image from among otherobjects in the image. Further, non-human objects can be identified, suchas the ball shown in FIG. 4A. After an individual is identified thesystem looks for any shapes that appear to be a human body (a head,torso, and two legs and arms), and then starts calculating things likehow those arms and legs are moving, where they can move (arms don't foldbackwards at the elbow, for example), and where they'll be in a fewmicroseconds. The identified locations can be mapped to a skeletalsystem.

During a motion associated with an activity, such as a shooting motionfor basketball, the orientation of the different body locations changerelative to one another during the motion. For example, in FIG. 4A, atthe beginning of a typical shot as shown 300 a, the individual's wrists,314 and 315, and elbows 306 and 312, are below the persons shoulders andnear the end of the shot as shown in 300 b, the individual's wrists andelbows are above the person's shoulders.

Using the identified body locations determined from the sensor systemand knowledge about a particular body motion that is expected to berepeated, one or more different orientations of the body locations canbe defined that tend to occur each time the motion is repeated. Thesystem can be configured to determine when one of the definedorientations has occurred.

As an example, for a shooting motion, a defined body orientation can bethe instance when one of the hands first reaches the top of the personshead during a shooting motion. Thus, the body orientation is detectedbased upon a specific relationship between a location on the hand and alocation at the top of the head. In another example, for a shootingmotion, the angle between the forearm and the upper arm can be tracked,i.e., the angle between the body segment including points 304 and 306and the body segment between points 306 and 314. In 300 a, this angle isabout ninety degrees. In 300 b, this angle is about 180 degrees whichcorresponds to the arm extending to a straight position. The system canbe configured to determine when a particular value of the angle, such as150 degrees, occurs. In this example, a specific relationship isspecified between three locations on the body.

In general, a specific relationship between two or more locations on thebody can be used to determine when a particular point in a motionoccurs. The specific relationship can be defined such that it will occurin each instance of the motion. For example, in the shooting motion, thespecific relationship might be a location on a person's hand crossinganother location on the body, such as on their chin or the top of theirhead. In one embodiment, when this relationship is not detected, themotion may not be used for the purposes of generating consistency data.For instance, in a shooting motion, if a location on a player's shootinghand is never detected to rise above their chest, then the shot may notbe used for the purposes of generating a motion consistency parameter.

The system may be configured not to look for the angle until itdetermines a shooting motion has been initiated. The initiation of theshooting motion might be defined as the hands moving upwards over somedistance or for some amount of time. Using this check, the system mayignore the situation where the player is merely extending their arms,such as when a ball is held below their waist. In general, the systemcan include logic for determining when a particular motion that is to betracked has started.

When a specified orientation of the body is detected, the system can beconfigured to generate data that can be used to generate a motionconsistency parameter. For example, in FIG. 4B, an angle 322 a betweenthree body points for a first body orientation is shown. The apex of theangle is at body location 306. The system can be configured to determinethis angle each time a specified orientation of the body is detectedduring a shooting motion. A motion consistency parameter can bedetermined based upon the variation of this angle over multiple shootingmotions. As described, in one embodiment, the variation can bequantified by determining a standard deviation.

In general, any three body points can be selected to define an angle.Further, multiple angles can be defined that utilize more than the threepoints. Thus, the example of angle 322 a is for the purposes ofillustration only and is not meant to be limiting.

As another example, the distance between two points can be measured eachtime a specific body orientation is detected. For instance, the distancebetween points 306 and 312 can be determined each time the orientationin 300 b is detected. Based upon this distance and data generated frommultiple instances of a captured shooing motion, a motion consistencyparameter can be generated. Again, any two points can be selected todetermine a distance that can be used in a motion consistency parameterand the example of points 306 and 312 is for illustrative purposes only.

In yet another example, a position of a single point may be determinedat a specific time in a motion, such as when a specific body orientationis detected during the motion. For example, in 300 b, the position oflocation 306 can be determined The position can be determined over manyinstances of the motion and then used to generate a motion consistencyparameter.

The system can be configured to detect when multiple body orientationsoccur during a motion, such as the shooting motion. Each time one of thebody orientations occur, the system can be configured to generateconsistency data that can be used to determine a motion consistencyparameter. For example, when the body orientation in 300 a is detected,angle 322 a can be determined and when the body orientation in 300 b isdetected angle 322 b can be determined The angles 322 a and 322 b, whichcan be determined over multiple shooting motions, can be used todetermine one or more motion consistency parameters. For instance,standard deviations can be determined for each of angles 322 a and 322 band then a single motion consistency parameter can be generated thatincludes both of these standard deviations.

When consistency data is generated at two or more times during a motion,then motion consistency parameters can be generated that involvedetermining changes between the measurements at the different times. Forexample, angle 322 a and angle 322 b can be determined at differenttimes during a shooting motion. Then, the difference between the anglescan be determined A motion consistency parameter can be generated thatis based upon the difference between the two angles.

In another embodiment, a time associated with a body motion can bedetermined For example, the system can be configured to identify whenthe body orientation in 300 a occurs and then to identify when the bodyorientation in 300 b occurs during a shooting motion. Amount of timethat occurs between the detection of the two body orientations can bedetermined Then, each time the shooting motion is repeated, an amount oftime between these two body orientations can be determined Based uponthe amount of time determined over many instances of the motion, amotion consistency parameter can be determined This type of parametermight be useful in a sport such as golf to characterize a consistency ofan aspect of their swing, such as the back swing.

As described above, the body motions can be used to estimate a forcethat can be imparted to an object. As an example, a position of thewrist 314 can be tracked over time during the shooting motion. In theinstance where a person is not holding a basketball but is pretending toshoot ball, the changes in position of the wrist can be used to estimatea magnitude of a force and a direction that would be imparted to anactual basketball.

As another example, if a person is swinging their arms in the manner ofhitting a tennis ball or a golf ball. A point or points on their wristor arms can be tracked to determine a force and direction of the forcethat would be imparted through their wrist. Then, a tennis racket or agolf club can be modeled as if it were extending from the wrist todetermine an amount of force that would be imparted to a hit object suchas golf ball or a tennis ball.

FIG. 5 is perspective drawing of video gaming system including a motioncapture sensor 410. An individual is standing in front of the sensor410. During the set-up process, the individual 402 may be directed tostand in a specific location, such as location 412.

The motion capture sensor may be capable of capturing stereoscopicimages. Further, the sensor 410 may be able to emit sound or light, suchas infrared light which can be bounced off the individual and detectedby the sensor. In one embodiment, the sensor 410 can be a Kinect™sensor. In another embodiment (not shown), a person can wear or hold adevice that is capable of capturing motion data. A Wii™ controller byNintendo™ or a mobile device with an accelerometer are examples of adevice a player could hold or even possibly wear.

An image display device 408 including a display screen 406 is coupled tothe video gaming system. An avatar 404 is output to the display screen.In one embodiment, motions captured by the sensor 410 can be used toanimate the avatar 404. The avatar 404 can be animated to mimic themovements of the individual 402.

After a video game is initiated, the individual 402 may make or one ormore different types of motions during the video game. For example, fora baseball game, the individual 402 may pretend to make a swingingmotion associated with swinging a bat at certain times and a pitchingmotion associated with throwing a ball at other times. For a basketballvideo game, the individual 402 may make shooting motions at differenttimes. In the video game, the shooting motions may be represented asbeing from different locations on a court. In a fighting video game, aperson may make kicking motions, punching motions or blocking motions atdifferent times.

Once a body motion is detected and the video gaming system determines itis a body motion for which an outcome is to be generated, the videogaming system can attempt to determine an outcome resulting from thecurrent body motion based upon data generated from the sensor system. Asdescribed above, the outcome can be determined based upon the currentinstance of the body motion and based upon data derived from previousdifferent instances where a similar body motion was generated. In oneembodiment, the data derived from the previous different instances canbe used to develop a measure of how consistent the individual makes themotion. A value for the measure can affect a probability of a desirableor non-desirable outcome occurring during the video game.

After the outcome for the current instance is determined, arepresentation of the outcome can be rendered to the display 406. Therepresentation can involve the avatar 404 mimicking to some degree themotions of the individual 402 and performing an action that theindividual is pretending to do. For instance, the individual 404 may bepretending to throw a baseball while the avatar 404 can be shownthrowing a baseball. If the individual is pretending to pitch, theavatar 404 may be shown throwing a ball or strike.

FIG. 6 is a block diagram of video gaming system 426 coupled to a motioncapture sensor 410 and a video display 406. The video gaming system 426is configured to predict outcomes using motion consistency parameters.The video gaming system 426 can include a processor, memory andcommunication interfaces, such as a network interface.

The system 426 can include logic 428 for analyzing data received fromsensor 410, such as logic for identifying and tracking various bodylocations on an individual during a motion used to play the game,determining the motion has been initiated, determining a particularpoint in the motion has occurred, determining a direction and a forceassociated with the body motion and determining data that can be used togenerate motion consistency parameter.

The system 426 can include logic for determining an outcome 430resulting from a motion captured by the system. The outcome can be adesirable or a non-desirable outcome. The probability of one or more ofthe outcomes occurring can depend on a motion consistency parameter. Thesystem can be configured to determine probability values and then basedupon the probabilities select an outcome. The outcome can be selectedusing a randomly generated number. The system can include logic 432 forrendering a selected outcome. The selected outcome can be shown on avideo display, such as 406.

The system 426 can include logic 434 for generating feedback associatedwith a captured body motion. The feedback can include information thathelps an individual to improve a body motion. For example, if theindividual is repeatedly making a shooting motion and the systemdetermines the person elbow position is not consistent, the system canprovide feedback that allows the user to focus on their elbow positionand improve the consistency of their body motion as related to the elbowposition.

The system 426 can include logic 436 for generating motion consistencyparameters. The generation of the parameters may include determiningwhether a particular instance of a body motion is to be used for thepurposes of generating a motion consistency parameter. For instance, ifa particular motion is determined to be invalid, then the system may notuse it in the determination of motion consistency parameter. The system426 can also be configured to reset motion consistency parameters suchthat its determination is based on more recently captured motion data.Further, the system can be configured to generate a new motionconsistency parameter each time a valid body motion is captured.

The various aspects, embodiments, implementations or features of thedescribed embodiments can be used separately or in any combination.Various aspects of the described embodiments can be implemented bysoftware, hardware or a combination of hardware and software. Thecomputer readable medium is any data storage device that can store datawhich can thereafter be read by a computer system. Examples of thecomputer readable medium include read-only memory, random-access memory,CD-ROMs, DVDs, magnetic tape and optical data storage devices. Thecomputer readable medium can also be distributed over network-coupledcomputer systems so that the computer readable code is stored andexecuted in a distributed fashion.

The foregoing description, for purposes of explanation, used specificnomenclature to provide a thorough understanding of the invention.However, it will be apparent to one skilled in the art that the specificdetails are not required in order to practice the invention. Thus, theforegoing descriptions of specific embodiments of the present inventionare presented for purposes of illustration and description. They are notintended to be exhaustive or to limit the invention to the precise formsdisclosed. It will be apparent to one of ordinary skill in the art thatmany modifications and variations are possible in view of the aboveteachings.

While the embodiments have been described in terms of several particularembodiments, there are alterations, permutations, and equivalents, whichfall within the scope of these general concepts. It should also be notedthat there are many alternative ways of implementing the methods andapparatuses of the present embodiments. It is therefore intended thatthe following appended claims be interpreted as including all suchalterations, permutations, and equivalents as fall within the truespirit and scope of the described embodiments.

1. An apparatus comprising for capturing body motion associated with aperson pretending to hit or launch an object: a processor, coupled to amemory and a sensor interface, the processor configured to: receive amotion consistency parameter wherein the motion consistency parameter isgenerated from data captured during a plurality of different instancesof the person attempting to repeat the body motion and wherein adetermining of the motion consistency parameter includes determining aposition of at least one point on a body of the person during each ofthe plurality of different instances of the body motion and determininga standard deviation using the determined positions; receive, via thesensor interface data from a sensor system, the data from the sensorsystem for a single instance of the person attempting to repeat the bodymotion while pretending to hit or launch the object; based upon the datafrom the single instance, in the processor, predict a magnitude of aforce and a direction of the force that would be imparted to a simulatedobject having a shape and mass of the object if it was actually hit orlaunched; based upon the predicted magnitude of the force and thepredicted direction of the force, determine whether a desired outcomefor the simulated object is possible; when the desired outcome isdetermined to be possible, determine whether the desired outcome or anon-desired outcome has occurred wherein a probability of the desiredoutcome occurring increases as the standard deviation associated withthe motion consistency parameter decreases; and control output to adisplay including one or more images of a simulated trajectory of thesimulated object for the single instance, wherein the simulatedtrajectory shows the determined desired outcome or the determinednon-desired outcome occurring for the simulated object.
 2. The apparatusof claim 1 wherein the sensor system includes a camera.
 3. The apparatusof claim 1 wherein the sensor system includes a device worn or held bythe person.
 4. The apparatus of claim 1 wherein the sensor systemincludes an accelerometer.
 5. The apparatus of claim 1 wherein theapparatus is a component of a video gaming system.
 6. The apparatus ofclaim 1 further comprising a network interface for communicating with aremote device.
 7. The apparatus of claim 1 wherein the simulatedtrajectory is output as part of a video game.
 8. The apparatus of claim1 wherein the processor is further configured to control output to thedisplay images of an avatar of the person making the body motion.
 9. Theapparatus of claim 1 wherein the processor is further configured todetermine whether the predicted magnitude of the estimated force and thepredicted direction of the force are within acceptable ranges for avalid body motion associated with the person pretending to hit or launchthe object; when the predicted magnitude of the estimated for and thepredicted direction of the force are not within the acceptable rangesthe valid body motion, determine the desired outcome is not possible andcontrol output to the display one or more images of a non-desiredoutcome associated with an non-valid body motion.
 10. The apparatus ofclaim 1, wherein the desired outcome is the simulated object stoppingwithin a defined area and the non-desired outcome is the simulatedobject stopping outside of a defined area.
 11. The apparatus of claim 1,wherein the object the person is pretending to hit or launch is a ball.